Search Results: "lamby"

14 July 2017

Chris Lamb: Installation birthday

Fancy receiving congratulations on the anniversary of when you installed your system? Installing the installation-birthday package on your Debian machines will celebrate each birthday of your system by automatically sending a message to the local system administrator. The installation date is based on the system installation time, not the package itself. installation-birthday is available in Debian 9 ("stretch") via the stretch-backports repository, as well as in the testing and unstable distributions:
$ apt install installation-birthday
Enjoy, and patches welcome. :)

30 June 2017

Chris Lamb: Free software activities in June 2017

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source. Multiple third-parties then can come to a consensus on whether a build was compromised or not. I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area. This month I:
  • Chaired our monthly IRC meeting. (Summary, logs, etc.)
  • Presented at Hong Kong Open Source Conference 2017.
  • Presented at LinuxCon China.
  • Submitted the following patches to fix reproducibility-related toolchain issues within Debian:
    • cracklib2: Ensuring /var/cache/cracklib/src-dicts are reproducible. (#865623)
    • fontconfig: Ensuring the cache files are reproducible. (#864082)
    • nfstrace: Make the PDF footers reproducible. (#865751)
  • Submitted 6 patches to fix specific reproducibility issues in cd-hit, janus, qmidinet, singularity-container, tigervnc & xabacus.
  • Submitted a wishlist request to the TeX mailing list to ensure that PDF files are reproducible even if generated from a difficult path after identifying underlying cause. (Thread)
  • Categorised a large number of packages and issues in the Reproducible Builds notes.git repository.
  • Worked on publishing our weekly reports. (#110, #111, #112 & #113)
  • Updated our website with 13 missing talks (e291180), updated the metadata for some existing talks (650a201) and added OpenEmbedded to the projects page (12dfcf0).

I also made the following changes to our tooling:
diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.


strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Add libarchive-cpio-perl with the !nocheck build profile. (01e408e)
  • Add dpkg-dev dependency build profile. (f998bbe)


Debian My activities as the current Debian Project Leader are covered in my "Bits from the DPL" email to the debian-devel-announce mailing list. However, I:
Debian LTS

This month I have been paid to work 16 hours hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 974-1 fixing a command injection vulnerability in picocom, a dumb-terminal emulation program.
  • Issued DLA 972-1 which patches a double-free vulnerability in the openldap LDAP server.
  • Issued DLA 976-1 which corrects a buffer over-read vulnerability in the yodl ("Your Own Document Language") document processor.
  • Issued DLA 985-1 to address a vulnerability in libsndfile (a library for reading/writing audio files) where a specially-crafted AIFF file could result in an out-of-bounds memory read.
  • Issued DLA 990-1 to fix an infinite loop vulnerability in the expat, an XML parsing library.
  • Issued DLA 999-1 for the openvpn VPN server if clients used a HTTP proxy with NTLM authentication, a man-in-the-middle attacker could cause the client to crash or disclose stack memory that was likely to contain the proxy password.

Uploads
  • bfs (1.0.2-1) New upstream release, add basic/smoke autopkgtests.
  • installation-birthday (5) Add some basic autopkgtest smoke tests and correct the Vcs- Git,Browser headers.
  • python-django:
    • 1:1.11.2-1 New upstream minor release & backport an upstream patch to prevent a test failure if the source is not writable. (#816435)
    • 1:1.11.2-2 Upload to unstable, use !nocheck profile for build dependencies that are only required for tests and various packaging updates.

I also made the following non-maintainer uploads (NMUs):
  • kluppe (0.6.20-1.1) Fix segmentation fault caused by passing a truncated pointer instead of a GtkType. (#863421)
  • porg (2:0.10-1.1) Fix broken LD_PRELOAD path for libporg-log.so. (#863495)
  • ganeti-instance-debootstrap (0.16-2.1) Fix "illegal option for fgrep" error by using "--" to escape the search needle. (#864025)
  • pavuk (0.9.35-6.1) Fix segmentation fault when opening the "Limitations" window due to pointer truncation in src/gtkmulticol.[ch]. (#863492)
  • timemachine (0.3.3-2.1) Fix two segmentation faults in src/gtkmeter.c and gtkmeterscale.c caused by passing a truncated pointers using guint instead of a GtkType. (#863420)
  • jackeq (0.5.9-2.1) Fix another segmentation fault caused by passing a truncated pointer instead of a GtkType. (#863416)

Debian bugs filed
  • debhelper: Don't run dh_installdocs if nodoc is specified in DEB_BUILD_PROFILES? (#865869)
  • python-blessed: Non-determistically FTBFS due to unreliable timing in tests. (#864337)
  • apt: Please print a better error message if zero certificates are loaded from the system CA store. (#866377)

31 May 2017

Chris Lamb: Free software activities in May 2017

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. (I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.) This month I:
I also made the following changes to our tooling:
diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • Don't fail when run under perversely-recursive input files. (#780761).

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Move from verbose_print to nonquiet_print so we print when normalising a file. This is so we can start to target the removal of strip-nondeterminism itself.
  • Only print log messages by default if the file was actually modified. (#863033)
  • Update package long descriptions to clarify that the tool itself is a temporary workaround. (#862029)


Debian My activities as the current Debian Project Leader are covered in my "Bits from the DPL" email to the debian-devel-announce list. However, I:
  • Represented Debian at the OSCAL 2017 in Tirana, Albania.
  • Attended the Reproducible Builds hackathon in Hamburg, Germany. (Report)
  • Finally, I attended Debian SunCamp 2017 in Lloret de Mar in Catalonia, Spain.

Patches contributed
  • xarchiver: Adding files to .tar.xz deletes existing content. (#862593)
  • screen-message: Please invert the default colours. (#862056)
  • fontconfig: fc-cache returns with exit code 0 on 256 errors. (#863427)
  • quadrapassel: Segfaults when unpausing a paused finished game. (#863106)
  • camping: Broken symlink. (#861040)
  • dns-root-data: Does not build if /bin/sh is Bash. (#862252)
  • dh-python: bit.ly link doesn't work anymore. (#863074)

Debian LTS

This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, adding links to upstream patches, etc.
  • Issued DLA 930-1 fixing a remote application crash vulnerability in libxstream-java, a Java library to serialize objects to XML and back again
  • Issued DLA 935-1 correcting a local denial of service vulnerability in lxterminal, the terminal emulator for the LXDE desktop environment.
  • Issued DLA 940-1 to remedy an issue in sane-backends which allowed remote attackers to obtain sensitive memory information via a crafted SANE_NET_CONTROL_OPTION packet.
  • Issued DLA 943-1 for the deluge bittorrent client to fix a directory traversal attack vulnerability in the web user interface.
  • Issued DLA 949-1 fixing an integer signedness error in the miniupnpc UPnP client that could allow remote attackers to cause a denial of service attack.
  • Issued DLA 959-1 for the libical calendaring library. A use-after-free vulnerability could allow remote attackers could cause a denial of service and possibly read heap memory via a specially crafted .ICS file.

Uploads
  • redis (3:3.2.9-1) New upstream release.
  • python-django:
    • 1:1.11.1-1 New upstream minor release.
    • 1:1.11.1-2 & 1:1.11.1-3 Add missing Build-Depends on libgdal-dev due to new GIS tests.
  • docbook-to-man:
    • 1:2.0.0-36 Adopt package. Apply a patch to prevent undefined behaviour caused by a memcpy(3) parameter overlap. (#842635, #858389)
    • 1:2.0.0-37 Install manpages using debian/docbook-to-man.manpages over manual calls.
  • installation-birthday Initial upload and misc. subsequent fixes.
  • bfs:
    • 1.0-3 Fix FTBFS on hurd-i386. (#861569)
    • 1.0.1-1 New upstream release & correct debian/watch file.

I also made the following non-maintainer uploads (NMUs):
  • ca-certificates (20161130+nmu1) Remove StartCom and WoSign certificates as they are now untrusted by the major browser vendors. (#858539)
  • sane-backends (1.0.25-4.1) Correct missing error handler in (generated) prerm script. (#862334)
  • seqan2 (2.3.1+dfsg-3.1) Fix broken /usr/bin/splazers symlink on 32-bit architectures. (#863669)
  • jackeq (0.5.9-2.1) Fix a segmentation fault caused by passing a truncated pointer instead of a GtkType. (#863416)
  • kluppe (0.6.20-1.1) Fix segmentation fault at startup. (#863421)
  • coyim (0.3.7-2.1) Skip tests that require internet access to avoid FTBFS. (#863414)
  • pavuk (0.9.35-6.1) Fix segmentation fault when opening "Limitations" window. (#863492)
  • porg (2:0.10-1.1) Fix broken LD_PRELOAD path. (#863495)
  • timemachine (0.3.3-2.1) Fix two segmentation faults caused by truncated pointers. (#863420)

Debian bugs filed
  • acct: Docs incorrectly installed to "accounting.html" directory. (#862180)
  • git-hub: Does not work with 2FA-enabled accounts. (#863265)
  • libwibble: Homepage and Vcs-Darcs fields are outdated. (#861673)



I additionally filed 2 bugs for packages that access the internet during build against flower and r-bioc-gviz.


I also filed 6 FTBFS bugs against cronutils, isoquery, libgnupg-interface-perl, maven-plugin-tools, node-dateformat, password-store & simple-tpm-pk11.

FTP Team

As a Debian FTP assistant I ACCEPTed 105 packages: boinc-app-eah-brp, debug-me, e-mem, etcd, fdroidcl, firejail, gcc-6-cross-ports, gcc-7-cross-ports, gcc-defaults, gl2ps, gnome-software, gnupg2, golang-github-dlclark-regexp2, golang-github-dop251-goja, golang-github-nebulouslabs-fastrand, golang-github-pkg-profile, haskell-call-stack, haskell-foundation, haskell-nanospec, haskell-parallel-tree-search, haskell-posix-pty, haskell-protobuf, htmlmin, iannix, libarchive-cpio-perl, libexternalsortinginjava-java, libgetdata, libpll, libtgvoip, mariadb-10.3, maven-resolver, mysql-transitional, network-manager, node-async-each, node-aws-sign2, node-bcrypt-pbkdf, node-browserify-rsa, node-builtin-status-codes, node-caseless, node-chokidar, node-concat-with-sourcemaps, node-console-control-strings, node-create-ecdh, node-create-hash, node-create-hmac, node-cryptiles, node-dot, node-ecc-jsbn, node-elliptic, node-evp-bytestokey, node-extsprintf, node-getpass, node-gulp-coffee, node-har-schema, node-har-validator, node-hawk, node-jsprim, node-memory-fs, node-pbkdf2, node-performance-now, node-set-immediate-shim, node-sinon-chai, node-source-list-map, node-stream-array, node-string-decoder, node-stringstream, node-verror, node-vinyl-sourcemaps-apply, node-vm-browserify, node-webpack-sources, node-wide-align, odil, onionshare, opensvc, otb, perl, petsc4py, pglogical, postgresql-10, psortb, purl, pymodbus, pymssql, python-decouple, python-django-rules, python-glob2, python-ncclient, python-parse-type, python-prctl, python-sparse, quoin-clojure, quorum, r-bioc-genomeinfodbdata, radlib, reprounzip, rustc, sbt-test-interface, slepc4py, slick-greeter, sparse, te923con, trabucco, traildb, typescript-types & writegood-mode. I additionally filed 6 RC bugs against packages that had incomplete debian/copyright files against: libgetdata, odil, opensvc, python-ncclient, radlib and reprounzip.

23 May 2017

Tianon Gravi: Debuerreotype

Following in the footsteps of one of my favorite Debian Developers, Chris Lamb / lamby (who is quite prolific in the reproducible builds effort within Debian), I ve started a new project based on snapshot.debian.org (time-based snapshots of the Debian archive) and some of lamby s work for creating reproducible Debian (debootstrap) rootfs tarballs. The project is named Debuerreotype as an homage to the photography roots of the word snapshot and the daguerreotype process which was an early method of taking photographs. The essential goal is to create photographs of a minimal Debian rootfs, so the name seemed appropriate (even if it s a bit on the mouthful side). The end-goal is to create and release Debian rootfs tarballs for a given point-in-time (especially for use in Docker) which should be fully reproducible, and thus improve confidence in the provenance of the Debian Docker base images. For more information about reproducibility and why it matters, see reproducible-builds.org, which has more thorough explanations of the why and how and links to other important work such as the reproducible builds effort in Debian (for Debian package builds). In order to verify that the tool actually works as intended, I ran builds against seven explicit architectures (amd64, arm64, armel, armhf, i386, ppc64el, s390x) and eight explicit suites (oldstable, stable, testing, unstable, wheezy, jessie, stretch, sid). I used a timestamp value of 2017-05-16T00:00:00Z, and skipped combinations that don t exist (such as wheezy on arm64) or aren t supported anymore (such as wheezy on s390x). I ran the scripts repeatedly over several days, using diffoscope to compare the results. While doing said testing, I ran across #857803, and added a workaround. There s also a minor outstanding issue with wheezy s reproducibility that I haven t had a chance to dig deep very deeply into yet (but it s pretty benign and Wheezy s LTS support window ends 2018-05-31, so I m not too stressed about it). I ve also packaged the tool for Debian, and submitted it into the NEW queue, so hopefully the FTP Masters will look favorably upon this being a tool that s available to install from the Debian archive as well. Anyhow, please give it a try, have fun, and as always, report bugs!

17 May 2017

Reproducible builds folks: Reproducible Builds: week 107 in Stretch cycle

Here's what happened in the Reproducible Builds effort between Sunday May 7 and Saturday May 13 2017: Report from Reproducible Builds Hamburg Hackathon We were 16 participants from 12 projects: 7 Debian, 2 repeatr.io, 1 ArchLinux, 1 coreboot + LEDE, 1 F-Droid, 1 ElectroBSD + privoxy, 1 GNU R, 1 in-toto.io, 1 Meson and 1 openSUSE. Three people came from the USA, 3 from the UK, 2 Finland, 1 Austria, 1 Denmark and 6 from Germany, plus we several guests from our gracious hosts at the CCCHH hackerspace as well as a guest from Australia We had four presentations: Some of the things we worked on: We had a Debian focussed meeting where we discussed a number of topics: And then we also had a lot of fun in the hackerspace, enjoying some of their gimmicks, such as being able to open physical doors with ssh or controlling light and music with an webbrowser without authentication (besides being in the right network). Not quite the hackathon (This wasn't the hackathon per-se, but some of us appreciated these sights and so we thought you would too.) Many thanks to: News and media coverage openSUSE has had a security breach in their infrastructure, including their build services. As of this writing, the scope and impact are still unclear, however the incident illustrates that no one should rely on being able to secure their infrastructure at all times. Reproducible Builds help mitigate this by allowing independent verification of build results, by parties that are unaffected by the compromise. (Whilst this can happen to anyone. Kudos to openSUSE for being open about it. Now let's continue working on Reproducible Builds everywhere!) On May 13th Chris Lamb gave a talk on Reproducible Builds at OSCAL 2017 in Tirana, Albania. OSCAL 2017 Toolchain bug reports and fixes Packages' bug reports Reviews of unreproducible packages 11 package reviews have been added, 2562 have been updated and 278 have been removed in this week, adding to our knowledge about identified issues. Most of the updates were to move ~1800 packages affected by the generic catch-all captures_build_path (out of ~2600 total) to the more specific gcc_captures_build_path, fixed by our proposed patches to GCC. 5 issue types have been updated: Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by: diffoscope development diffoscope development continued on the experimental branch: strip-nondeterminism development reprotest development trydiffoscope development Misc. This week's edition was written by Ximin Luo, Holger Levsen and Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

30 April 2017

Chris Lamb: Free software activities in April 2017

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area. This month I:
I also made the following changes to diffoscope, our recursive and content-aware diff utility used to locate and diagnose reproducibility issues:
  • New features:
    • Add support for comparing Ogg Vorbis files. (0436f9b)
  • Bug fixes:
    • Prevent a traceback when using --new-file with containers. (#861286)
    • Don't crash on invalid archives; print a useful error instead. (#833697).
    • Don't print error output from bzip2 call. (21180c4)
  • Cleanups:
    • Prevent abstraction-level violations by defining visual diff support on Presenter classes. (7b68309)
    • Show Debian packages installed in test output. (c86a9e1)


Debian
Debian LTS

This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 882-1 for the tryton-server general application platform to fix a path suffix injection attack.
  • Issued DLA 883-1 for curl preventing a buffer read overrun vulnerability.
  • Issued DLA 884-1 for collectd (a statistics collection daemon) to close a potential infinite loop vulnerability.
  • Issued DLA 885-1 for the python-django web development framework patching two open redirect & XSS attack issues.
  • Issued DLA 890-1 for ming, a library to create Flash files, closing multiple heap-based buffer overflows.
  • Issued DLA 892-1 and DLA 891-1 for the libnl3/libnl Netlink protocol libraries, fixing integer overflow issues which could have allowed arbitrary code execution.

Uploads
  • redis (4:4.0-rc3-1) New upstream RC release.
  • adminer:
    • 4.3.0-2 Fix debian/watch file.
    • 4.3.1-1 New upstream release.
  • bfs:
    • 1.0-1 Initial release.
    • 1.0-2 Drop fstype tests as they rely on /etc/mtab being available. (#861471)
  • python-django:
    • 1:1.10.7-1 New upstream security release.
    • 1:1.11-1 New upstream stable release to experimental.

I sponsored the following uploads: I also performed the following QA uploads:
  • gtkglext (1.2.0-7) Correct installation location of gdkglext-config.h after "Multi-Archification" in 1.2.0-5. (#860007)
Finally, I made the following non-maintainer uploads (NMUs):
  • python-formencode (1.3.0-2) Don't ship files in /usr/lib/python 2.7,3 /dist-packages/docs. (#860146)
  • django-assets (0.12-2) Patch pytest plugin to check whether we are running in a Django context, otherwise we can break unrelated testsuites. (#859916)


FTP Team

As a Debian FTP assistant I ACCEPTed 155 packages: aiohttp-cors, bear, colorize, erlang-p1-xmpp, fenrir, firejail, fizmo-console, flask-ldapconn, flask-socketio, fontmanager.app, fonts-blankenburg, fortune-zh, fw4spl, fzy, gajim-antispam, gdal, getdns, gfal2, gmime, golang-github-go-macaron-captcha, golang-github-go-macaron-i18n, golang-github-gogits-chardet, golang-github-gopherjs-gopherjs, golang-github-jroimartin-gocui, golang-github-lunny-nodb, golang-github-markbates-goth, golang-github-neowaylabs-wabbit, golang-github-pkg-xattr, golang-github-siddontang-goredis, golang-github-unknwon-cae, golang-github-unknwon-i18n, golang-github-unknwon-paginater, grpc, grr-client-templates, gst-omx, hddemux, highwayhash, icedove, indexed-gzip, jawn, khal, kytos-utils, libbloom, libdrilbo, libhtml-gumbo-perl, libmonospaceif, libpsortb, libundead, llvm-toolchain-4.0, minetest-mod-homedecor, mini-buildd, mrboom, mumps, nnn, node-anymatch, node-asn1.js, node-assert-plus, node-binary-extensions, node-bn.js, node-boom, node-brfs, node-browser-resolve, node-browserify-des, node-browserify-zlib, node-cipher-base, node-console-browserify, node-constants-browserify, node-delegates, node-diffie-hellman, node-errno, node-falafel, node-hash-base, node-hash-test-vectors, node-hash.js, node-hmac-drbg, node-https-browserify, node-jsbn, node-json-loader, node-json-schema, node-loader-runner, node-miller-rabin, node-minimalistic-crypto-utils, node-p-limit, node-prr, node-sha.js, node-sntp, node-static-module, node-tapable, node-tough-cookie, node-tunein, node-umd, open-infrastructure-storage-tools, opensvc, openvas, pgaudit, php-cassandra, protracker, pygame, pypng, python-ase, python-bip32utils, python-ltfatpy, python-pyqrcode, python-rpaths, python-statistics, python-xarray, qtcharts-opensource-src, r-cran-cellranger, r-cran-lexrankr, r-cran-pwt9, r-cran-rematch, r-cran-shinyjs, r-cran-snowballc, ruby-ddplugin, ruby-google-protobuf, ruby-rack-proxy, ruby-rails-assets-underscore, rustc, sbt, sbt-launcher-interface, sbt-serialization, sbt-template-resolver, scopt, seqsero, shim-signed, sniproxy, sortedcollections, starjava-array, starjava-connect, starjava-datanode, starjava-fits, starjava-registry, starjava-table, starjava-task, starjava-topcat, starjava-ttools, starjava-util, starjava-vo, starjava-votable, switcheroo-control, systemd, tilix, tslib, tt-rss-notifier-chrome, u-boot, unittest++, vc, vim-ledger, vis, wesnoth-1.13, wolfssl, wuzz, xandikos, xtensor-python & xwallpaper. I additionally filed 14 RC bugs against packages that had incomplete debian/copyright files against getdns, gfal2, grpc, mrboom, mumps, opensvc, python-ase, sniproxy, starjava-topcat, starjava-ttools, unittest++, wolfssl, xandikos & xtensor-python.

16 April 2017

Chris Lamb: Elected Debian Project Leader

I'd like to thank the entire Debian community for choosing me to represent them as the next Debian Project Leader. I would also like to thank Mehdi for his tireless service and wish him all the best for the future. It is an honour to be elected as the DPL and I am humbled that you would place your faith and trust in me. You can read my platform here.

31 March 2017

Chris Lamb: Free software activities in March 2017

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area. This month I:
I also made the following changes to our tooling:
diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • New features/optimisations:
    • Extract squashfs archive in one go rather than per-file, speeding up ISO comparison by ~10x.
    • Add support for .docx and .odt files via docx2txt & odt2txt. (#859056).
    • Add support for PGP files via pgpdump. (#859034).
    • Add support for comparing Pcap files. (#858867).
    • Compare GIF images using gifbuild. (#857610).
  • Bug fixes:
    • Ensure that we really are using ImageMagick and not the GraphicsMagick compatibility layer. (#857940).
    • Fix and add test for meaningless 1234-content metadata when introspecting archives. (#858223).
    • Fix detection of ISO9660 images processed with isohybrid.
    • Skip icc tests if the Debian-specific patch is not present. (#856447).
    • Support newer versions of cbfstool to avoid test failures. (#856446).
    • Update the progress bar prior to working to ensure filename is in sync.
  • Cleanups:
    • Use /usr/share/dpkg/pkg-info.mk over manual calls to dpkg-parsechangelog in debian/rules.
    • Ensure tests and the runtime environment can locate binaries in /usr/sbin (eg. tcpdump).

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Fix a possible endless loop while stripping .ar files due to trusting the file's own file size data. (#857975).
  • Add support for testing files we should reject and include the filename when evaluating fixtures.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Add support for Format: 1.0. (#20).
  • Don't parse Format: header as the source package version. (#21).
  • Show the reproducible status of packages.


Debian


I submitted my platform for the 2017 Debian Project Leader Elections. This was subsequently covered on LWN and I have been participating in the discussions on the debian-vote mailing list since then.


Debian LTS

This month I have been paid to work 14.75 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 848-1 for the freetype font library fixing a denial of service vulnerability.
  • Issued DLA 851-1 for wget preventing a header injection attack.
  • Issued DLA 863-1 for the deluge BitTorrent client correcting a cross-site request forgery vulnerability.
  • Issued DLA 864-1 for jhead (an EXIF metadata tool) patching an arbitrary code execution vulnerability.
  • Issued DLA 865-1 for the suricata intrusion detection system, fixing an IP protocol matching error.
  • Issued DLA 871-1 for python3.2 fixing a TLS stripping vulnerability in the smptlib library.
  • Issued DLA 873-1 for apt-cacher preventing a HTTP response splitting vulnerability.
  • Issued DLA 876-1 for eject to prevent an issue regarding the checking of setuid(2) and setgid(2) return values.

Uploads
  • python-django:
    • 1:1.10.6-1 New upstream bugfix release.
    • 1:1.11~rc1-1 New upstream release candidate.
  • redis:
    • 3:3.2.8-2 Avoid conflict between RuntimeDirectory and tmpfiles.d(5) both attempting to create /run/redis with differing permissions. (#856116)
    • 3:3.2.8-3 Revert the creation of a /usr/bin/redis-check-rdb to /usr/bin/redis-server symlink to avoid a dangling symlink if only the redis-tools package is installed. (#858519)
  • gunicorn 19.7.0-1 & 19.7.1-1 New upstream releases.
  • adminer 4.3.0-1 New upstream release.

Finally, I also made the following non-maintainer uploads (NMUs):


FTP Team

As a Debian FTP assistant I ACCEPTed 121 packages: 4pane, adql, android-platform-system-core, android-sdk-helper, braillegraph, deepnano, dh-runit, django-auth-ldap, django-dirtyfields, drf-extensions, gammaray, gcc-7, gnome-keysign, golang-code.gitea-sdk, golang-github-bluebreezecf-opentsdb-goclient, golang-github-bsm-redeo, golang-github-cupcake-rdb, golang-github-denisenkom-go-mssqldb, golang-github-exponent-io-jsonpath, golang-github-facebookgo-ensure, golang-github-facebookgo-freeport, golang-github-facebookgo-grace, golang-github-facebookgo-httpdown, golang-github-facebookgo-stack, golang-github-facebookgo-subset, golang-github-go-openapi-loads, golang-github-go-openapi-runtime, golang-github-go-openapi-strfmt, golang-github-go-openapi-validate, golang-github-golang-geo, golang-github-gorilla-pat, golang-github-gorilla-securecookie, golang-github-issue9-assert, golang-github-issue9-identicon, golang-github-jaytaylor-html2text, golang-github-joho-godotenv, golang-github-juju-errors, golang-github-kisielk-gotool, golang-github-kubernetes-gengo, golang-github-lpabon-godbc, golang-github-lunny-log, golang-github-makenowjust-heredoc, golang-github-mrjones-oauth, golang-github-nbutton23-zxcvbn-go, golang-github-neelance-sourcemap, golang-github-ngaut-deadline, golang-github-ngaut-go-zookeeper, golang-github-ngaut-log, golang-github-ngaut-pools, golang-github-ngaut-sync2, golang-github-optiopay-kafka, golang-github-quobyte-api, golang-github-renstrom-dedent, golang-github-sergi-go-diff, golang-github-siddontang-go, golang-github-smartystreets-go-aws-auth, golang-github-xanzy-go-cloudstack, golang-github-xtaci-kcp, golang-github-yohcop-openid-go, graywolf, haskell-raaz, hfst-ospell, hikaricp, iptraf-ng, kanboard-cli, kcptun, kreport, libbluray, libcatmandu-store-elasticsearch-perl, libcsfml, libnet-prometheus-perl, libosmocore, libpandoc-wrapper-perl, libseqlib, matrix-synapse, mockldap, nfs-ganesha, node-buffer, node-pako, nose-el, nvptx-tools, nx-libs, open-ath9k-htc-firmware, pagein, paleomix, pgsql-ogr-fdw, profanity, pyosmium, python-biotools, python-django-extra-views, python-django-otp, python-django-push-notifications, python-dnslib, python-gmpy, python-gmpy2, python-holidays, python-kanboard, python-line-profiler, python-pgpy, python-pweave, python-raven, python-xapian-haystack, python-xopen, r-cran-v8, repetier-host, ruby-jar-dependencies, ruby-maven-libs, ruby-psych, ruby-retriable, seafile-client, spyder-unittest, stressant, systray-mdstat, telegram-desktop, thawab, tigris, tnseq-transit, typesafe-config, vibe.d, x2goserver & xmlrpc-c. I additionally filed 14 RC bugs against packages that had incomplete debian/copyright files against: golang-github-cupcake-rdb, golang-github-sergi-go-diff, graywolf, hfst-ospell, libbluray, pgsql-ogr-fdw, python-gmpy, python-gmpy2, python-pgpy, python-xapian-haystack, repetier-host, telegram-desktop, tigris & xmlrpc-c.

25 March 2017

Bits from Debian: Debian Project Leader elections 2017

It's that time of year again for the Debian Project: the elections of its Project Leader! The Project Leader position is described in the Debian Constitution. Two Debian Developers run this year to become Project Leader: Mehdi Dogguy, who has held the office for the last year, and Chris Lamb. We are in the middle of the campaigning period that will last until the end of April 1st. The candidates and Debian contributors are already engaging in debates and discussions on the debian-vote mailing list. The voting period starts on April 2nd, and during the following two weeks, Debian Developers can vote to choose the person that will fit that role for one year. The results will be published on April 16th with the term for new the project leader starting the following day.

28 February 2017

Chris Lamb: Free software activities in February 2017

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. (I have been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.) This month I:
I also made the following changes to our tooling:
diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • New features:
    • Add a machine-readable JSON output format. (Closes: #850791).
    • Add an --exclude option. (Closes: #854783).
    • Show results from debugging packages last. (Closes: #820427).
    • Extract archive members using an auto-incrementing integer avoiding the need to sanitise filenames. (Closes: #854723).
    • Apply --max-report-size to --text output. (Closes: #851147).
    • Specify <html lang="en"> in the HTML output. (re. #849411).
  • Bug fixes:
    • Fix errors when comparing directories with non-directories. (Closes: #835641).
    • Device and RPM fallback comparisons require xxd. (Closes: #854593).
    • Fix tests that call xxd on Debian Jessie due to change of output format. (Closes: #855239).
    • Add missing Recommends for comparators. (Closes: #854655).
    • Importing submodules (ie. parent.child) will attempt to import parent. (Closes: #854670).
    • Correct logic of module_exists ensuring we correctly skip the debian.deb822 tests when python3-debian is not installed. (Closes: #854745).
    • Clean all temporary files in the signal handler thread instead of attempting to pass the exception back to the main thread. (Closes: #852013).
    • Fix behaviour of setting report maximums to zero (ie. no limit).
  • Optimisations:
    • Don't uselessly run xxd(1) on non-directories.
    • No need to track libarchive directory locations.
    • Optimise create_limited_print_func.
  • Tests:
    • When comparing two empty directories, ensure that the mtime of the directory is consistent to avoid non-deterministic failures.
    • Ensure we can at least import the "deb_fallback" and "rpm_fallback" modules.
    • Add test for symlink differing in destination.
    • Add tests for --progress, --status-fd and profiling output options as well as the Deb Changes,Buildinfo,Dsc and RPM fallback comparisons.
    • Add get_data and @skip_unless_module_exists test helpers.
    • Mark impossible-to-reach code to improve test coverage.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Drop raw_text fields now as we've moved these to Amazon S3.
  • Drop storage of Installed-Build-Depends and subsequently-orphaned Binary package instances to recover diskspace.

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Print log entry when fixing a file. (Closes: #777239).
  • Run our entire testsuite in autopkgtests, not just the first test. (Closes: #852517).
  • Don't test for stat(2)'s blksize and block attributes. (Closes: #854937).
  • Use error() from Dh_Lib.pm over "manual" die().


Debian
Debian LTS

This month I have been paid to work 13 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 817-1 for libphp-phpmailer, correcting a local file disclosure vulnerability where insufficient parsing of HTML messages could potentially be used by attacker to read a local file.
  • Issued DLA 826-1 for wireshark which fixes a denial of service vulnerability in wireshark, where a malformed NATO Ground Moving Target Indicator Format ("STANAG 4607") capture file could cause a memory exhausion/infinite loop.

Uploads
  • python-django (1:1.11~beta1-1) New upstream beta release.
  • redis (3:3.2.8-1) New upstream release.
  • gunicorn (19.6.0-11) Use $ misc:Pre-Depends to populate Pre-Depends for dpkg-maintscript-helper.
  • dh-virtualenv (1.0-1~bpo8+1) Upload to jessie-backports.

I sponsored the following uploads: I also performed the following QA uploads:
  • dh-kpatches (0.99.36+nmu4) Make kernel kernel builds reproducible.
Finally, I made the following non-maintainer uploads:
  • cpio (2.12+dfsg-3) Remove rmt.8.gz to prevent a piuparts error.
  • dot-forward (1:0.71-2.2) Correct a FTBFS; we don't install anything to /usr/sbin, so use GNU Make's $(wildcard ..) over the shell's own * expansion.


FTP Team

As a Debian FTP assistant I ACCEPTed 116 packages: autobahn-cpp, automat, bglibs, bitlbee, bmusb, bullet, case, certspotter, checkit-tiff, dash-el, dash-functional-el, debian-reference, el-x, elisp-bug-hunter, emacs-git-messenger, emacs-which-key, examl, genwqe-user, giac, golang-github-cloudflare-cfssl, golang-github-docker-goamz, golang-github-docker-libnetwork, golang-github-go-openapi-spec, golang-github-google-certificate-transparency, golang-github-karlseguin-ccache, golang-github-karlseguin-expect, golang-github-nebulouslabs-bolt, gpiozero, gsequencer, jel, libconfig-mvp-slicer-perl, libcrush, libdist-zilla-config-slicer-perl, libdist-zilla-role-pluginbundle-pluginremover-perl, libevent, libfunction-parameters-perl, libopenshot, libpod-weaver-section-generatesection-perl, libpodofo, libprelude, libprotocol-http2-perl, libscout, libsmali-1-java, libtest-abortable-perl, linux, linux-grsec, linux-signed, lockdown, lrslib, lua-curses, lua-torch-cutorch, mariadb-10.1, mini-buildd, mkchromecast, mocker-el, node-arr-exclude, node-brorand, node-buffer-xor, node-caller, node-duplexer3, node-ieee754, node-is-finite, node-lowercase-keys, node-minimalistic-assert, node-os-browserify, node-p-finally, node-parse-ms, node-plur, node-prepend-http, node-safe-buffer, node-text-table, node-time-zone, node-tty-browserify, node-widest-line, npd6, openoverlayrouter, pandoc-citeproc-preamble, pydenticon, pyicloud, pyroute2, pytest-qt, pytest-xvfb, python-biomaj3, python-canonicaljson, python-cgcloud, python-gffutils, python-h5netcdf, python-imageio, python-kaptan, python-libtmux, python-pybedtools, python-pyflow, python-scrapy, python-scrapy-djangoitem, python-signedjson, python-unpaddedbase64, python-xarray, qcumber, r-cran-urltools, radiant, repo, rmlint, ruby-googleauth, ruby-os, shutilwhich, sia, six, slimit, sphinx-celery, subuser, swarmkit, tmuxp, tpm2-tools, vine, wala & x265. I additionally filed 8 RC bugs against packages that had incomplete debian/copyright files against: checkit-tiff, dash-el, dash-functional-el, libcrush, libopenshot, mkchromecast, pytest-qt & x265.

4 February 2017

Chris Lamb: The ChangeLog #237: Reproducible Builds and Secure Software

I recently appeared on the Changelog podcast to talk about the Reproducible Builds project:


Whilst I am an avid podcast listener, this was actually my first appearance on one. It was an curious and somewhat disconcerting feeling to be "just" talking to Adam and Jerod in the moment yet knowing all the time that anything and everything I said would be distributed more widely in the future.

31 January 2017

Chris Lamb: Free software activities in January 2017

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. (I have previously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.) This month I:
I also made the following changes to our tooling:
diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • Comparators:
    • Display magic file type when we know the file format but can't find file-specific details. (Closes: #850850).
    • Ensure our "APK metadata" file appears first, fixing non-deterministic tests. (998b288)
    • Fix APK extration with absolute filenames. (Closes: #850485).
    • Don't error if directory containing ELF debug symbols already exists. (Closes: #850807).
    • Support comparing .ico files (Closes: #850730).
    • If we don't have a tool (eg. apktool), don't blow up when attempting to unpack it.
  • Output formats:
    • Add Markdown output format. (Closes: #848141).
    • Add RestructuredText output format.
    • Use an optimised indentation routine throughout all presenters.
    • Move text presenter to use the Visitor pattern.
    • Correctly escape value of href="" elements (re. #849411).
  • Tests:
    • Prevent FTBFS by loading fixtures as UTF-8 in case surrounding terminal is not Unicode-aware. (Closes: #852926).
    • Skip tests if binutils can't handle the object file format. (Closes: #851588).
    • Actually compare the output of text/ReST/Markdown formats to fixtures.
    • Add tests for: Comparing two empty directories, HTML output, image.ICOImageFile, --html-dir, --text-color & no arguments (beyond the filenames) emits the text output.
  • Profiling:
    • Count the number of calls, not just the total time.
    • Skip as much profiling overhead when not enabled for a ~2% speedup.
  • Misc:
    • Alias an expensive Config() lookup for a 10% optimisation.
    • Avoid expensive regex creation until we actually need it, speeding up diff parsing by 2X.
    • Use Pythonic logging functions based on __name__, etc.
    • Drop milliseconds from logging output.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Store files directly onto S3.
  • Drop big unique_together index to save disk space.
  • Show SHA256 checksums where space permits.


Debian LTS

This month I have been paid to work 12.75 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 773-1 for python-crypto fixing a vulnerability where calling AES.new with an invalid parameter could crash the Python interpreter.
  • Issued DLA 777-1 for libvncserver addressing two heap-based buffer overflow attacks based on invalid FramebufferUpdate data.
  • Issued DLA 778-1 for pcsc-lite correcting a use-after-free vulnerability.
  • Issued DLA 795-1 for hesiod which fixed a weak SUID check as well as removed the hard-coding of a fallback domain if the configuration file could not be found.
  • Issued DLA 810-1 for libarchive fixing a heap buffer overflow.

Uploads
  • python-django:
    • 1:1.10.5-1 New upstream stable release.
    • 1:1.11~alpha1-1 New upstream experimental release.
  • gunicorn (19.6.0-10) Moved debian/README.Debian to debian/NEWS so that the recent important changes will be displayed to users when upgrading to stretch.
  • redis:
    • 3:3.2.6-2 & 4:4.0-rc2-2 Tidy patches and rename RunTimeDirectory to RuntimeDirectory in .service files. (Closes: #850534)
    • 3:3.2.6-3 Remove a duplicate redis-server binary by symlinking /usr/bin/redis-check-rdb. This was found by the dedup service.
    • 3:3.2.6-4 Expand the documentation in redis-server.service and redis-sentinel.service regarding the default hardening options and how, in most installations, they can be increased.
    • 3:3.2.6-5, 3:3.2.6-6, 4:4.0-rc2-3 & 4:4.0-rc2-4 Add taskset calls to try and avoid build failures due to parallelism in upstream test suite.

I also made the following non-maintainer uploads:
  • cpio:
    • 2.12+dfsg-1 New upstream release (to experimental), refreshing all patches, etc.
    • 2.12+dfsg-2 Add missing autoconf to Build-Depends.
  • xjump (2.7.5-6.2) Make the build reproducible by passing -n to gzip calls in debian/rules. (Closes: #777354)
  • magicfilter (1.2-64.1) Make the build reproducible by passing -n to gzip calls in debian/rules. (Closes: #777478)



11 January 2017

Reproducible builds folks: Reproducible Builds: week 89 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday January 1 and Saturday January 7 2017: GSoC and Outreachy updates Toolchain development Packages reviewed and fixed, and bugs filed Chris Lamb: Dhole: Reviews of unreproducible packages 13 package reviews have been added, 4 have been updated and 6 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been added/updated: Upstreaming of reproducibility fixes Merged: Opened: Weekly QA work During our reproducibility testing, the following FTBFS bugs have been detected and reported by: diffoscope development diffoscope 67 was uploaded to unstable by Chris Lamb. It included contributions from :
[ Chris Lamb ]
* Optimisations:
  - Avoid multiple iterations over archive by unpacking once for an ~8X
    runtime optimisation.
  - Avoid unnecessary splitting and interpolating for a ~20X optimisation
    when writing --text output.
  - Avoid expensive diff regex parsing until we need it, speeding up diff
    parsing by 2X.
  - Alias expensive Config() in diff parsing lookup for a 10% optimisation.
* Progress bar:
  - Show filenames, ELF sections, etc. in progress bar.
  - Emit JSON on the the status file descriptor output instead of a custom
    format.
* Logging:
  - Use more-Pythonic logging functions and output based on __name__, etc.
  - Use Debian-style "I:", "D:" log level format modifier.
  - Only print milliseconds in output, not microseconds.
  - Print version in debug output so that saved debug outputs can standalone
    as bug reports.
* Profiling:
  - Also report the total number of method calls, not just the total time.
  - Report on the total wall clock taken to execute diffoscope, including
    cleanup.
* Tidying:
  - Rename "NonExisting" -> "Missing".
  - Entirely rework diffoscope.comparators module, splitting as many separate
    concerns into a different utility package, tidying imports, etc.
  - Split diffoscope.difference into diffoscope.diff, etc.
  - Update file references in debian/copyright post module reorganisation.
  - Many other cleanups, etc.
* Misc:
  - Clarify comment regarding why we call python3(1) directly. Thanks to J r my
    Bobbio <lunar@debian.org>.
  - Raise a clearer error if trying to use --html-dir on a file.
  - Fix --output-empty when files are identical and no outputs specified.
[ Reiner Herrmann ]
* Extend .apk recognition regex to also match zip archives (Closes: #849638)
[ Mattia Rizzolo ]
* Follow the rename of the Debian package "python-jsbeautifier" to
  "jsbeautifier".
[ siamezzze ]
* Fixed no newline being classified as order-like difference.
reprotest development reprotest 0.5 was uploaded to unstable by Chris Lamb. It included contributions from:
[ Ximin Luo ]
* Stop advertising variations that we're not actually varying.
  That is: domain_host, shell, user_group.
* Fix auto-presets in the case of a file in the current directory.
* Allow disabling build-path variations. (Closes: #833284)
* Add a faketime variation, with NO_FAKE_STAT=1 to avoid messing with
  various buildsystems. This is on by default; if it causes your builds
  to mess up please do file a bug report.
* Add a --store-dir option to save artifacts.
Other contributions (not yet uploaded): reproducible-builds.org website development tests.reproducible-builds.org Misc. This week's edition was written by Chris Lamb, Holger Levsen and Vagrant Cascadian, reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

9 January 2017

Guido G nther: Debian Fun in December 2016

Debian LTS November marked the 20th month I contributed to Debian LTS under the Freexian umbrella. I had 8 hours allocated which I used by: Other Debian stuff Some other Free Software activites

7 January 2017

Thorsten Alteholz: My Debian Activities in December 2016

FTP assistant This month I marked 367 packages for accept and rejected 45 packages. This time I only sent 10 emails to maintainers asking questions. Debian LTS This was my thirtieth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 13.50h. During that time I did uploads of Other stuff The Debian Med Advent Calendar was really successful this year. As announced in [1] this year the second highest number of bugs has been closed during tht bug squashing:
year number of bugs closed
2011 63
2012 28
2013 73
2014 5
2015 150
2016 95
Well done everybody who participated! In December I also uploaded new upstream versions of duktape, fixed bugs in openzwave, did a binary upload for mpb on mipsel, sponsored openzwave-controlpanel, sidedoor and printrun.
Thanks to lamby that openzwave-controlpanel and sidedoor even made it into Stretch. Last but not least I want to wish everybody a Happy New Year. [1] https://lists.debian.org/debian-med/2016/12/msg00180.html

31 December 2016

Chris Lamb: Free software activities in December 2016

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. This month:
I also made the following changes to our tooling:
diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • Optimisations:
    • Avoid unnecessary string manipulation writing --text output (~20x speedup).
    • Avoid n iterations over archive files (~8x speedup).
    • Don't analyse .deb s twice when comparing .changes files (2x speedup).
    • Avoid shelling out to colordiff by implementing color support directly.
    • Memoize calls to distutils.spawn.find_executable to avoid excessive stat(1) syscalls.
  • Progress bar:
    • Show current file / ELF section under analysis etc. in progress bar.
    • Move the --status-fd output to use JSON and to include the current filename.
  • Code tidying:
    • Split out the try.diffoscope.org client so that it can be released separately on PyPI.
    • Completely rework the diffoscope and diffoscope.comparators modules, grouping similar utilities into their own modules, etc.
  • Miscellaneous:
    • Update dex_expected_diffs test to ensure compatibility with enjarify 1.0.3.
    • Ensure that running from Git will always use that checkout's Python modules.
    • Add a simple profiling framework.

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.

  • Makefile.PL: Change NAME argument to a Perl package name.
  • Ensure our binaries are available in autopkgtest tests.

try.diffoscope.org

trydiffoscope is a web-based version of the diffoscope in-depth and content-aware diff utility. Continued thanks to Bytemark for sponsoring the hardware.

  • Show progress bar and position in queue, etc. (#25 & #26)
  • Promote command-line client with PyPI instructions.
  • Increase comparison time limit to 90 seconds.

buildinfo.debian.net

buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

  • Added support for version 0.2 .buildinfo files. (#15)

Debian
Debian LTS

This month I have been paid to work 13 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 733-1 for openafs, fixing an information leak vulnerability. Due to incomplete initialization or clearing of reused memory, directory objects could contain 'dead' directory entry information.
  • Issued DLA 734-1 for mapserver closing an information leakage vulnerability.
  • Issued DLA 737-1 for roundcube preventing arbitrary remote code execution by sending a specially crafted email.
  • Issued DLA 738-1 for spip patching a cross-site scripting (XSS) vulnerability.
  • Issued DLA 740-1 for libgsf fixing a null pointer deference exploit via a crafted .tar file.

Debian Uploads
  • redis:
    • 3.2.5-5 Add RunTimeDirectory=redis to systemd .service files.
    • 3.2.5-6 Add missing Depends on lsb-base for /lib/lsb/init-functions usage in redis-sentinel's initscript.
    • 3.2.6-1 New upstream release.
    • 4.0-1 & 4.0-rc2-1 New upstream experimental releases.
  • aptfs: 0.9-1 & 0.10-1 New upstream releases.


Debian FTP Team

As a Debian FTP assistant I ACCEPTed 107 packages: android-platform-libcore, compiz, debian-edu, dehydrated, dh-cargo, gnome-shell-extension-pixelsaver, golang-1.8, golang-github-btcsuite-btcd-btcec, golang-github-elithrar-simple-scrypt, golang-github-pelletier-go-toml, golang-github-restic-chunker, golang-github-weaveworks-mesh, golang-google-genproto, igmpproxy, jimfs, kpmcore, libbio-coordinate-perl, libdata-treedumper-oo-perl, libdate-holidays-de-perl, libpgobject-type-bytestring-perl, libspecio-library-path-tiny-perl, libterm-table-perl, libtext-hogan-perl, lighttpd, linux, linux-signed, llmnrd, lua-geoip, lua-sandbox-extensions, lua-systemd, node-cli-cursor, node-command-join, node-death, node-detect-indent, node-domhandler, node-duplexify, node-end-of-stream, node-first-chunk-stream, node-from2, node-glob-stream, node-has-binary, node-inquirer, node-interpret, node-is-negated-glob, node-is-unc-path, node-lazy-debug-legacy, node-lazystream, node-load-grunt-tasks, node-merge-stream, node-object-assign-sorted, node-orchestrator, node-pkg-up, node-resolve-from, node-resolve-pkg, node-rx, node-sorted-object, node-stream-shift, node-streamtest, node-string.prototype.codepointat, node-strip-bom-stream, node-through2-filter, node-to-absolute-glob, node-unc-path-regex, node-vinyl, openzwave, openzwave-controlpanel, pcb-rnd, pd-upp, pg-partman, postgresql-common, pybigwig, python-acora, python-cartopy, python-codegen, python-efilter, python-flask-sockets, python-intervaltree, python-jsbeautifier, python-portpicker, python-pretty-yaml, python-protobix, python-sigmavirus24-urltemplate, python-sqlsoup, python-tinycss, python-watson-developer-cloud, python-zc.customdoctests, python-zeep, r-cran-dbitest, r-cran-dynlm, r-cran-mcmcpack, r-cran-memoise, r-cran-modelmetrics, r-cran-plogr, r-cran-prettyunits, r-cran-progress, r-cran-withr, ruby-clean-test, ruby-gli, ruby-json-pure, ruby-parallel, rustc, sagemath, sbuild, scram, sidedoor, toolz & yabasic. I additionally filed 4 RC bugs against packages that had incomplete debian/copyright files against jimfs, compiz, python-efilter & ruby-json-pure.

30 December 2016

Chris Lamb: My favourite books of 2016

Whilst I managed to read almost sixty books in 2016 here are ten of my favourites in no particular order. Disappointments this year include Stewart Lee's Content Provider (nothing like his stand-up), Christopher Hitchens' And Yet (his best essays are already published) and Heinlein's Stranger in a Strange Land (great exposition, bizarre conclusion). The worst book I finished, by far, was Mark Edward's Follow You Home.





https://images-eu.ssl-images-amazon.com/images/P/B010EAQLV2.01._PC__.jpg Animal QC Gary Bell, QC Subtitled My Preposterous Life, this rags-to-riches story about a working-class boy turned eminent lawyer would be highly readable as a dry and factual account but I am compelled to include it here for its extremely entertaining style of writing. Full of unsurprising quotes that take one unaware: would you really expect a now-Queen's Counsel to "heartily suggest that if you find yourself suffering from dysentery in foreign climes you do not medicate it with lobster thermidor and a bottle of Ecuadorian red?" A real good yarn.
https://images-eu.ssl-images-amazon.com/images/P/B0196HJ6OS.01._PC__.jpg So You've Been Publically Shamed Jon Ronson The author was initially recommended to me by Brad but I believe I started out with the wrong book. In fact, I even had my doubts about this one, prematurely judging from the title that it was merely cashing-in on a fairly recent internet phenomenon like his more recent shallow take on Trump and the alt-Right but in the end I read Publically Shamed thrice in quick succession. I would particularly endorse the audiobook version: Ronson's deadpan drawl suits his writing perfectly.
https://images-eu.ssl-images-amazon.com/images/P/B00IX49OS4.01._PC__.jpg The Obstacle is the Way Ryan Holiday Whilst everyone else appears to be obligated to include Ryan's recent Ego is the Enemy in their Best of 2016 lists I was actually taken by his earlier "introduction by stealth" to stoic philosophy. Certainly not your typical self-help book, this is "a manual to turn to in troubling times". Returning to this work at least three times over the year even splashing out on the audiobook at some point I feel like I learned a great deal, although it is now difficult to pinpoint exactly what. Perhaps another read in 2017 is thus in order
https://images-eu.ssl-images-amazon.com/images/P/071563335X.01._PC__.jpg Layer Cake J.J. Connolly To judge a book in comparison to the film is to do both a disservice, but reading the book of Layer Cake really underscored just how well the film played to the strengths of that medium. All of the aspects that would not have worked had been carefully excised from the screenplay, ironically leaving more rewarding "layers" for readers attempting the book. A parallel adaption here might be No Country for Old Men - I would love to read (or write) a comparative essay between these two adaptions although McCarthy's novel is certainly the superior source material.
https://images-eu.ssl-images-amazon.com/images/P/B00G1SRB6Q.01._PC__.jpg Lying Sam Harris I've absorbed a lot of Sam Harris's uvre this year in the form of his books but moreover via his compelling podcast. I'm especially fond of Waking Up on spirituality without religion and would rank that as my favourite work of his. Lying is a comparatively short read, more of a long essay in fact, where he argues that we can radically simplify our lives by merely telling the truth in situations where others invariably lie. Whilst it would take a brave soul to adopt his approach his case is superlatively well-argued and a delight to read.
https://images-eu.ssl-images-amazon.com/images/P/0140442103.01._PC__.jpg Letters from a Stoic Seneca

Great pleasure is to be found not only in keeping up an old and established friendship but also in beginning and building up a new one. Reading this in a beautifully svelte hardback, I tackled a randomly-chosen letter per day rather than attempting to read it cover-to-cover. Breaking with a life-long tradition, I even decided to highlight sections in pen so I could return to them at ease. I hope it's not too hackneyed to claim I gained a lot from "building up" a relationship with this book. Alas, it is one of those books that is too easy to recommend given that it might make one appear wise and learned, but if you find yourself in a slump, either in life or in your reading habits, it certainly has my approval.


https://images-eu.ssl-images-amazon.com/images/P/B00BHD3TIE.01._PC__.jpg Solo: A James Bond Novel William Boyd I must have read all of the canonical Fleming novels as a teenager and Solo really rewards anyone who has done so. It would certainly punish anyone expecting a Goldeneye or at least be a little too foreign to be enjoyed. Indeed, its really a pastiche of these originals, both in terms of the time period, general tone (Bond is more somber; more vulnerable) and in various obsessions of Fleming's writing, such as the overly-detailed description of the gambling and dining tables. In this universe, 007's restaurant expenses probably contributed signifcantly to the downfall of the British Empire, let alone his waistline. Bond flicking through a ornithological book at one point was a cute touch
https://images-eu.ssl-images-amazon.com/images/P/B019MMUA8S.01._PC__.jpg The Subtle Art of Not Giving A F*ck Mark Manson Certainly a wildcard to include here and not without its problems, The Subtle Art is a curious manifesto on how to approach life. Whilst Manson expouses an age-old philosophy of grounding yourself and ignoring the accumulation of flatscreen TVs, etc. he manages to do so in a fresh and provocative "21st-centry gonzo" style. Highly entertaining, at one point the author posits an alternative superhero ("Disappointment Panda") that dishes out unsolicited and uncomfortable truths to strangers before simply walking away: "You know, if you make more money, that s not going to make your kids love you," or: "What you consider friendship is really just your constant attempts to impress people." Ouch.
https://images-eu.ssl-images-amazon.com/images/P/B004ZLS5RK.01._PC__.jpg The Fourth Protocol Frederick Forsyth I have a crystal-clear memory from my childhood of watching a single scene from a film in the dead of night: Pierce Brosnan sets a nuclear device to detonate after he can get away but a double-crossing accomplice surreptitiously brings the timetable forward in order that the bomb also disposes of him Anyway, at some point whilst reading The Fourth Protocol it dawned on me that this was that book. I might thus be giving the book more credit due to this highly satisfying connection but I think it stands alone as a superlative political page-turner and is still approachable outside the machinations of the Cold War.
https://images-eu.ssl-images-amazon.com/images/P/B003IDMUSG.01._PC__.jpg The Partner John Grisham After indulging in a bit too much non-fiction and an aborted attempt at The Ministry of Fear, I turned to a few so-called lower-brow writers such as Jeffrey Archer, etc. However, it was The Partner that turned out to be a real page-turner for somewhat undefinable reasons. Alas, it appears the rest of the author's output is unfortunately in the same vein (laywers, etc.) so I am hesitant to immediately begin others but judging from various lists online I am glad I approached this one first.
https://images-eu.ssl-images-amazon.com/images/P/B00D3J2QKC.01._PC__.jpg Shogun: The First Novel of the Asian saga James Clavell Despite its length, I simply couldn't resist returning to Shogun this year although it did fatigue me to the point that I have still yet to commence on its sequel, Tai-Pan. Like any good musical composition, one is always rewarded by returning to a book and I took great delight in uncovering more symbolism throughout (such as noticing that one of the first words Blackthorne learns in Japanese is "truth") but also really savouring the tragic arcs that run throughout the novel, some beautiful phrases ("The day seemed to lose its warmth ") and its wistful themes of inevitability and karma.

19 December 2016

Chris Lamb: 10 years of Debian

Today marks the 10-year anniversary of my first contribution to Debian GNU/Linux. I will not recount the full history here but my first experience with Debian was a happy accident. I had sent off for a 5-CD set of Red Hat from The Linux Emporium only to discover I lacked the required 12MB of RAM. Annoyed, I reached for the Debian "potato" CD that was included gratis in my order due to it being outdated at the time Fast-forwarding a few years, whilst my first contribution was trivial, it was Thomas Bushnell's infectious enthusiasm that led me to contribute more, eventually becoming a Google Summer of Code student under Daniel Baumann, and finally becoming an official Debian Developer in September 2008 with Thomas Viehmann as my Application Manager. (Some things may never change, however I still struggle with the bug tracker's control@ interface.)
The response I got to my patch always reminds me of the irrational power of providing attibution. I've always liked to tell myself I'm above such vanities but perhaps the truly mature approach would be to accept that ego is part of the human condition and as a community take steps to avoid handicapping ourselves by underestimating the value of "trivialities" such as having one's name listed. I've since been fascinated by the number of maintainers who do not attribute patches in changelogs, especially from newcomers or when the changes are non-trivial a handful in particular have stung me fairly deeply. I would certainly concede that it adds nothing technical and can even be distracting, but it seems a reasonable concession that dramatically increases the chance of future efforts or, frankly, is simply a kindly gesture of thanks and good will. Given our level of technical expertise, I fear we regularly suffer from not having sufficient empathy for newcomers or first-time users who lack the context or orientation that we possess. Anyway, here's to another ten

30 November 2016

Chris Lamb: Free software activities in November 2016

Here is my monthly update covering what I have been doing in the free software world (previous month):
Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to permit verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

This month:

My work in the Reproducible Builds project was also covered in our weekly reports. (#80, #81, #82 #83.

Toolchain issues I submitted the following patches to fix reproducibility-related toolchain issues with Debian:

strip-nondeterminism

strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.


jenkins.debian.net

jenkins.debian.net runs our comprehensive testing framework.

  • buildinfo.debian.net has moved to SSL. (ac3b9e7)
  • Submit signing keys to keyservers after generation. (bdee6ff)
  • Various cosmetic changes, including
    • Prefer if X not in Y over if not X in Y. (bc23884)
    • No need for a dictionary; let's just use a set. (bf3fb6c)
    • Avoid DRY violation by using a for loop. (4125ec5)

I also submitted 9 patches to fix specific reproducibility issues in apktool, cairo-5c, lava-dispatcher, lava-server, node-rimraf, perlbrew, qsynth, tunnelx & zp.

Debian

Debian LTS This month I have been paid to work 11 hours on Debian Long Term Support (LTS). In that time I did the following:
  • "Frontdesk" duties, triaging CVEs, etc.
  • Issued DLA 697-1 for bsdiff fixing an arbitrary write vulnerability.
  • Issued DLA 705-1 for python-imaging correcting a number of memory overflow issues.
  • Issued DLA 713-1 for sniffit where a buffer overflow allowed a specially-crafted configuration file to provide a root shell.
  • Issued DLA 723-1 for libsoap-lite-perl preventing a Billion Laughs XML expansion attack.
  • Issued DLA 724-1 for mcabber fixing a roster push attack.

Uploads
  • redis:
    • 3.2.5-2 Tighten permissions of /var/ lib,log /redis. (#842987)
    • 3.2.5-3 & 3.2.5-4 Improve autopkgtest tests and install upstream's MANIFESTO and README.md documentation.
  • gunicorn (19.6.0-9) Adding autopkgtest tests.
  • libfiu:
    • 0.94-1 Add autopkgtest tests.
    • 0.95-1, 0.95-2 & 0.95-3 New upstream release and improve autopkgtest coverage.
  • python-django (1.10.3-1) New upstream release.
  • aptfs (0.8-3, 0.8-4 & 0.8-5) Adding and subsequently improving the autopkgtext tests.


I performed the following QA uploads:


Finally, I also made the following non-maintainer uploads:
  • libident (0.22-3.1) Move from obsolete Source-Version substvar to binary:Version. (#833195)
  • libpcl1 (1.6-1.1) Move from obsolete Source-Version substvar to binary:Version. (#833196)
  • pygopherd (2.0.18.4+nmu1) Move from obsolete Source-Version substvar to $ source:Version . (#833202)


RC bugs


I also filed 59 FTBFS bugs against arc-gui-clients, asyncpg, blhc, civicrm, d-feet, dpdk, fbpanel, freeciv, freeplane, gant, golang-github-googleapis-gax-go, golang-github-googleapis-proto-client-go, haskell-cabal-install, haskell-fail, haskell-monadcatchio-transformers, hg-git, htsjdk, hyperscan, jasperreports, json-simple, keystone, koji, libapache-mod-musicindex, libcoap, libdr-tarantool-perl, libmath-bigint-gmp-perl, libpng1.6, link-grammar, lua-sql, mediatomb, mitmproxy, ncrack, net-tools, node-dateformat, node-fuzzaldrin-plus, node-nopt, open-infrastructure-system-images, open-infrastructure-system-images, photofloat, ppp, ptlib, python-mpop, python-mysqldb, python-passlib, python-protobix, python-ttystatus, redland, ros-message-generation, ruby-ethon, ruby-nokogiri, salt-formula-ceilometer, spykeviewer, sssd, suil, torus-trooper, trash-cli, twisted-web2, uftp & wide-dhcpv6.

FTP Team

As a Debian FTP assistant I ACCEPTed 70 packages: bbqsql, coz-profiler, cross-toolchain-base, cross-toolchain-base-ports, dgit-test-dummy, django-anymail, django-hstore, django-html-sanitizer, django-impersonate, django-wkhtmltopdf, gcc-6-cross, gcc-defaults, gnome-shell-extension-dashtodock, golang-defaults, golang-github-btcsuite-fastsha256, golang-github-dnephin-cobra, golang-github-docker-go-events, golang-github-gogits-cron, golang-github-opencontainers-image-spec, haskell-debian, kpmcore, libdancer-logger-syslog-perl, libmoox-buildargs-perl, libmoox-role-cloneset-perl, libreoffice, linux-firmware-raspi3, linux-latest, node-babel-runtime, node-big.js, node-buffer-shims, node-charm, node-cliui, node-core-js, node-cpr, node-difflet, node-doctrine, node-duplexer2, node-emojis-list, node-eslint-plugin-flowtype, node-everything.js, node-execa, node-grunt-contrib-coffee, node-grunt-contrib-concat, node-jquery-textcomplete, node-js-tokens, node-json5, node-jsonfile, node-marked-man, node-os-locale, node-sparkles, node-tap-parser, node-time-stamp, node-wrap-ansi, ooniprobe, policycoreutils, pybind11, pygresql, pysynphot, python-axolotl, python-drizzle, python-geoip2, python-mockupdb, python-pyforge, python-sentinels, python-waiting, pythonmagick, r-cran-isocodes, ruby-unicode-display-width, suricata & voctomix-outcasts. I additionally filed 4 RC bugs against packages that had incomplete debian/copyright files against node-cliui, node-core-js, node-cpr & node-grunt-contrib-concat.

21 November 2016

Reproducible builds folks: Reproducible Builds: week 82 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday November 13 and Saturday November 19 2016: Media coverage Elsewhere in Debian Documentation update Packages reviewed and fixed, and bugs filed Reviews of unreproducible packages 43 package reviews have been added, 4 have been updated and 12 have been removed in this week, adding to our knowledge about identified issues. 2 issue types have been updated: 4 issue types have been added: Weekly QA work During our reproducibility testing, some FTBFS bugs have been detected and reported by: strip-nondeterminism development disorderfs development debrebuild development debrebuild is new tool proposed by HW42 and josch (see #774415: "From srebuild sbuild-wrapper to debrebuild"). debrepatch development debrepatch is a set of scripts that we're currently developing to make it easier to track unapplied patches. We have a lot of those and we're not always sure if they still work. The plan is to set up jobs to automatically apply old reproducibility patches to newer versions of packages and notify the right people if they don't apply and/or no longer make the package reproducible. debpatch is a component of debrepatch that applies debdiffs to Debian source packages. In other words, it is to debdiff(1) what patch(1) is to diff(1). It is a general tool that is not specific to Reproducible Builds. This week, Ximin Luo worked on making it more "production-ready" and will soon submit it for inclusion in devscripts. reprotest development Ximin Luo significantly improved reprotest, adding presets and auto-detection of which preset to use. One can now run e.g. reprotest auto . or reprotest auto $pkg_$ver.dsc instead of the long command lines that were needed before. He also made it easier to set up build dependencies inside the virtual server and made it possible to specify pre-build dependencies that reprotest itself needs to set up the variations. Previously one had to manually edit the virtual server to do that, which was not very usable to humans without an in-depth knowledge of the building process. These changes will be tested some more and then released in the near future as reprotest 0.4. tests.reproducible-builds.org Misc. This week's edition was written by Chris Lamb, Holger Levsen, Ximin Luo and reviewed by a bunch of Reproducible Builds folks on IRC.

Next.

Previous.